Feature Transfer Learning for Speech Emotion Recognition

Speech Emotion Recognition (SER) has achieved some substantial progress in the past few decades since the dawn of emotion and speech research. In many aspects, various research efforts have been made in an attempt to achieve human-like emotion recognition performance in real-life settings. However, with the availability of speech data obtained from different devices and varied acquisition conditions, SER systems are often faced with scenarios, where the intrinsic distribution mismatch between the training and the test data has an adverse impact on these systems. To address this issue, this thesis makes use of autoencoders as an expressive learner to introduce a set of novel feature transfer learning algorithms. They are based on the goal to achieve a matched feature space representation for the target and source sets while ensuring source domain knowledge transfer. Partly inspired by the recent successes of feature learning, this thesis first incorporates sparse autoencoders into semi-supervised feature transfer learning. Furthermore, in the unsupervised setting, i.e., without the availability of any labeled target data in the training phase, this thesis takes advantage of denoising autoencoders, shared-hiddenlayer autoencoders, adaptive denoising autoencoders, extreme learning machine autoencoders, and subspace learning with denoising autoencoders, for feature transfer learning. Experimental results are presented on a wide range of emotional speech databases , demonstrating the advantages of the proposed algorithms over other modern transfer learning methods. Besides normal phonated speech, these transfer learning methods are also evaluated on whispered speech emotion recognition, which shows that these methods can be applied to create a recognition model owing a completely trainable architecture that can adapt it to a range of speech modalities.